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26 November 1975 


MEMORANDUM FOR THE RECORD Yale 


SUBJECT: Project ORACLE,[ sss Preshipment 


Acceptance Test 


The[———s—s——s*di Preshippment Acceptance Test was 


held for seven successive days over the period 19 November 


through 25 November 1975. There were 20 test periods, 
all of them failed. 


The test exercised both the software and hardware 
of the Mass Storage System in conjunction with a host 
computer, an IBM 370/155. The functions tested were 


mutually agreed to by[_ J and the Agency on 3 July 1975. 


All individual tests but one were submitted in advance 


of the test period to[___—_—_s—*|JThe test set was delivered 


on 25 August 1975 and a revised set was delivered on 

20 October 1975. The tests actually run during the 
period differed from those previously submitted to 

only in that the data content of the files had been 
altered. Altering the data content had no impact on 
the functional aspects of the test, it simply precluded 
any kind of pre-arrangement on the part of [4 


The acceptance test was conducted by the Agency, 


[| personnel were present as observers. The Agency 


decided what functions were to be tested in a given 
period and in what sequence the individual tests were 
to execute. Agency personnel alone made the decision 
as to whether a test period was a failure or a success. 
Criteria used to determine success or failure of fun- 
ctional tests were based solely on the Mass Storage 
System Design (Specification) dated 19 March 1975. 
Whenever a test period ended because the system stopped 
processing, it was simply declared a failure. The 
Agency did not attempt to diagnose the cause as either 


hardware or software. 


During each test period, the Agency had complete 
control of the system. At the end of a test period 
control was returned tol | ‘After each test period, 
all events were reviewed, an evaluation of the test 
was made and then presented to[___} +~Finally, a 


brief written summary of the test period was given to 
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[|which served to notify them of the results. We 


also retained the blow-by-blow detail in the form of 
machine console logs, printer output from the host 
computer, and handwritten logs of events that took 
place during each test. A new test period would be 
started only whenL____]declared the system was ready 
and returned control to the Agency. Any changes made 
b were noted so that we could determine if 

any tests needed to be rerun. 


It 1S important to note that the nature of all 
the failures were clear-cut. It was not necessary 
to make any intensive studies of the specifications 
to probe for subtle interpretations. The failures 
encountered during the tests were both hardware and 
software related. [ _sJdid not question any of the 
test evaluations. 


There were two serious types of failures. The 
first was the inability of the Mass Storage System 
to move some files from disk to the TBM Tape. These 
same files could be manipulated successfully from the 
host computer. The second failure was the inability 
of the system to move some files from TBM Tape to disk. 
The specification and the system design assumes this 
latter problem will occur once for each 2.5 billion 
characters of data. The test results showed a rate 
of 50 occurrences for each 2.5 billion characters. 


Because Of the short duration of the acceptance 
test and the limited number of functions being tested, 
the system should look better than it really is. Using 
this for a guideline and considering the poor quality 
revealed by the test, it is my judgement that [] 
is not over half way done with the Mass Storage System. 


COTR 
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